An innovative concept map approach for improving students' learning performance with an instant feedback mechanism

Concept maps have been widely employed for helping students organise their knowledge as well as evaluating their knowledge structures in a wide range of subject matters. Although researchers have recognised concept maps as being an important educational tool, past experiences have also revealed the difficulty of evaluating the correctness of a concept map. It usually takes days or weeks for teachers to manually evaluate the concept maps developed by students; consequently, the students cannot receive timely feedback from the teachers, which not only affects their learning schedules, but also significantly influences the students’ learning achievements. In this paper, a computerbased concept map-oriented learning strategy with real-time assessment and feedback is proposed in order to cope with the problems mentioned above. Our approach provides immediate evaluation of concept maps and gives also real-time feedback to the students. An experiment has been conducted to evaluate the effectiveness of this new strategy in comparison with the conventional computer-based concept map approach. It is found that our innovative approach can be significantly beneficial to promote learning achievements as well as the learning attitudes of students. Introduction Novak and Gowin (1984) pointed out that instructors should teach new knowledge based on learners’ original cognitive structures, so that they can connect old cognitive structures with new ones to form meaningful learning. During the past 10 years, concept maps have been used as a tool for supporting meaningful learning in science teaching and for helping students and experts to represent and visualise their knowledge structures (Novak, 2002). Through concept maps, learners are able to externalise their original knowledge and combine it with new one for rearranging and internalising both the old and new knowledge (Erdogan, 2009; Lim, Lee & Grabowski, 2009; Trundle & Bell, 2010). Akinsanya and Williams (2004) stated that concept maps are effective tools for students to clarify their knowledge structures; consequently, they have encouraged educators to take concept maps into consideration when designing their course plans. British Journal of Educational Technology Vol 43 No 2 2012 217–232 doi:10.1111/j.1467-8535.2010.01167.x © 2011 The Authors. British Journal of Educational Technology © 2011 BERA. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. Owing to the popularity of computer technology, the use of computerised concept maps has become popular. Several previous studies have shown that the students who learned with computerised concept map systems had better learning achievements than those who learned with traditional approaches (Kim & Olaciregui, 2008). The advantages of computerised concept maps include the ease of making corrections, the flexibility of presenting content and the availability of promoting interactions among teachers and students (Liu, Chen & Chang, 2010; Reader & Hammond, 1994; Shin, Deno, Robinson & Marston, 2000). In the past decade, a number of studies have attempted to develop computer-assisted learning or assessment systems combined with the concept map strategy. For example, Hwang (2003) and Hwang, Tseng and Hwang (2008) employed concept maps to develop learning systems, which can diagnose students’ learning problems and provide personalised learning guidance, as well as appropriate learning materials. In the meantime, several studies have highlighted the importance of giving prompt feedback to students (Denton, Madden, Roberts & Rowe, 2008; Draper, 2009; Jordan & Mitchell, 2009; Li, Liu & Steckelberg, 2010). For example, Narciss and Huth (2006) claimed that immediate feedback is obviously beneficial for learning achievement and motivation. Moreover, Carroll’s (1963) mastery theory also emphasised the advantages of providing immediate feedback. In a mastery theory-oriented learning activity, teachers give instructions based on the teaching goal, try every possible way to assist students in repeated practice and allow students to have enough practice in order to achieve the learning goal. An objective evaluation standard is needed for assessing the learning achievements of students and providing feedback to them based on the assessment results during the repeated practice process that enables students to learn continually until they reach a certain level of achievement (Chu, Hwang & Huang, 2010; Johnson, Perry & Shamir, 2010; Lalley & Gentile, 2009; Panjaburee, Hwang, Triampo & Shih, 2010). Consequently, it can be seen that the provision of instant feedback is very important to the students in enabling them to learn continually until reaching a certain level of achievement or organising their knowledge in a more meaningful way, in particular, for those complex learning tasks such as the development of concept maps. Unfortunately, although learning with concept maps seems to be promising, researchers have pointed out a critical problem for using concept maps to support learning; ie, an extra load is created for the teachers who have to evaluate the maps developed by individual students. Usually it takes days or weeks for teachers to complete the evaluation of the concept maps; therefore, the students cannot receive feedback immediately (Ingeç, 2009). Denton et al (2008) indicated that instructors confront a number of challenges if they wish to return meaningful and instant feedback to individual students. Therefore, giving instant and meaningful feedback for complex learning tasks such as the development of concept maps has become an important and challenging issue (Denton et al, 2008; Hwang, Chu, Shih, Huang & Tsai, 2010). To cope with this problem, this study proposes a computer-based concept map-oriented learning approach with real-time assessment and instant feedback. With the prompts and feedback, the students can reflect on and make modifications to their concept maps immediately; ie, they do not need to wait for the evaluation results from the teachers before proceeding with their learning process. Thus, the research questions that guided the work presented in this paper can be formulated as following: 1. Are the learning achievements of the students who learn with this proposed approach (concept map strategy with instant feedback) significantly higher than those who learn with the traditional concept map approach? 2. Are the learning attitudes of the students who learn with the new approach more positive than those who learn with the traditional concept map approach? 218 British Journal of Educational Technology Vol 43 No 2 2012 © 2011 The Authors. British Journal of Educational Technology © 2011 BERA. 3. Are the technology-acceptance degrees of the students who learn with the new approach significantly higher than those who learn with the traditional concept map approach? The paper is organised as follows: the next section describes our approach that combines concept maps with an instant feedback mechanism and the technology to support this process. The following section presents the experimental design including a detailed description of the participants, treatments and measuring tools. Analysis and discussions of the results introduced in the previous section are presented in order to get a better understanding of the different aspects and outcomes of the learning process. Finally, in the last section, concluding remarks are drawn and future steps are put forward. A concept map approach with an instant feedback mechanism In this study, a concept map approach with an instant feedback mechanism is proposed. Guided by this innovative perspective, a web-based learning system is developed for helping the students efficiently create and modify their concept maps based on the knowledge and experiences gained from the practice, such that their learning achievements can be improved. System framework and functions We have called the system that combines concept map-oriented learning with an instant feedback mechanism the ‘Interactive Concept Map-oriented Learning System’ (ICMLS). Figure 1 illustrates the different components of ICMLS, in which CmapTools, developed by the Institute for Human and Machine Cognition (IHMC) of the Florida University System (Novak & Cañas, 2006), is adopted as the concept map editing tool to work with the assessment and feedback system in the learning activities. CmapTools is a well-recognised tool for constructing, navigating and sharing knowledge models represented as concept maps. It enables learners to construct concept maps in personal computers and share them on servers via the Internet (http://cmap.ihmc.us/ conceptmap.html). The concept maps developed by the students are stored in a database. Via accessing the database, the students can revise their maps. When submitting a new version of a concept map, the Figure 1: The different components of the computerised concept map system with an instant feedback mechanism Concept map approach with an instant feedback mechanism 219 © 2011 The Authors. British Journal of Educational Technology © 2011 BERA. students will receive feedback as well as supplementary materials from the assessment and feedback system, as shown in Figure 1. To create or modify concept maps, students use a concept keyword list compiled by the teachers. With the acquired knowledge, the students employ CmapTools to concretely present the relevant knowledge with concept maps, which enables the students to deliberate on the connections among these concepts, and reorganise the links of each concept. When the students finish drawing their concept maps, the maps will be retrieved automatically by the assessment and feedback system for evaluation. Based on the evaluation results, the system will provide feedback, supplementary information, as well as it will retrieve the concept map scores to the students. The feedback includes the score and a set of comments on the structure of the concept map developed by individual students, such as ‘There is a missing notion related to Concept A’ and ‘There is a missing connection related to Concept A and some other Concept’. On the other hand, the supplementary information is a set of learning materials related to the missing or incorrect concepts/connections. When a student’s concept map is closer to the target concept map developed by the teacher, the feedback contains fewer comments and a higher score; in the meantime, less corresponding supplementary information is provided. Figure 2 shows the interface of the assessment and feedback system. The ‘historic concept map area’ shows the version number, scores and the link to each version of the concept map. By clicking on the links, the students can review their old concept maps on the Web, which enables them to compare their new versions with the old ones. The concept maps with higher scores indicate a higher degree of completeness and accuracy. Clicking the ‘score’ button in the historic Link to the supplementary information Comments about the missing or incorrect parts in the concept map Link to the developed concept map Version number